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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address « 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• tf the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )IE Responsive to communication(s) filed on 6/13/05 & 12/13/04 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-1 1 and 18-33 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 1-11 and 18-33 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) Q The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1 ) H Notice of References Cited (PTO-892) 4) D Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) S Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) D Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date 6/13/05 . 6) Q Other: . 
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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1-11 and 18-33 have been considered but 
are moot in view of the new ground(s) of rejection. 

Claim Objections 

2. Claim 1 is objected to because of the following informalities: "when printed on one the 
more pages" (line 9) should be changed to -when printed on the one or more pages-. 
Correction is required. 

Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1, 4-7, 10, 11, 18, 21-24, 27-29, 32 and 33 are rejected under 35 U.S.C- 103(a) 
as being unpatentable over Schelling et al (US 5,706,097), Young-Il Choi et al (article 
entitled "An Integrated Data Model and a Query Language for Content-Based Retrieval of 
Video) and Lowitz et al (US 5,485,554). 
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Claims L LL 28, 29, 32, and 33: 

Schelling discloses a method for generating an index print (paper document) for a 
multimedia document storing multimedia information including video, audio, graphic, and text 
information. A user may select representations of the information for retrieving information of 
his interest. Schelling teaches identifiers (clues) that help the user to identify relevant 
information. Schelling teaches that users may print the multimedia information on a paper 
document (abstract; col. 1, lines 58-67; col. 2, lines 1-14, 43-67; col. 3, lines 1-25; col. 4, lines 2- 
7). Schelling does not teach accepting user input to identify a concept of interest. However, 
Choi discloses a video data model that integrates feature-based model and annotation based 
model for improving content-based retrieval of video data that can be used in multimedia 
applications and which supports free annotations. Choi teaches the annotation-based model uses 
keywords to represent the video contents (abstract; page 192, first and second paragraph; 
page 197, pg. 4.3; fig. 2). Therefore, it would have been obvious to one having ordinary skill in 
the art at the time the invention was made to modify Schelling to include Choi's teaching of 
accepting user input of keywords to identify a concept of interest because it provides automated 
analysis, content-based video browsing, and an efficient way for users to retrieve specified video 
data. Schelling/Choi does not teach that information to be relevant to the concept of interest is 
annotated when printed on a page. However, Lowitz discloses a method and system for printing 
video images on a printable medium (abstract; col. 1, lines 8-15, 35-38; col. 2, lines 6-34; col. 3, 
lines 23-60; col. 4, lines 8-12). Lowitz teaches that video information can encoded in barcode or 
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the printable media (col 2, lines 35-49); a portion of a frame of video data may be selected and 
stored as printable image data and processed to correlate the portion of the frame using an 
identifier; the identifier is a tag or mark associated with a specific frame and it can be printed 
together with the video data (col. 2, lines 50-62; col. 8, lines 23-64); and media data (video, 
audio) can include annotations anywhere on the printable medium (col. 11, lines 44-67; col. 12, 
lines 1-20; col. 14, lines 57-67; col. 15, lines 1-14). Therefore, it would have been obvious to 
one ordinarily skilled in the art at the time the invention was made to modify Schelling/Choi's 
method of generating a paper document for multimedia information including annotations 
because they provide the user with enhanced information that is easy and quick to assimilate. 
Claims 4 and 21: 

See claim 1. Schelling teaches a method of generating an index print (printable 
representation) for a multimedia document having multimedia information (text, audio, video), 
(abstract; col. 1, lines 58-67; col. 2, lines 1-20). The method has layout information for printing 
the printable representation of the first and second type on a paper medium (col. 3, lines 1-40). 
Claims 5 and 22: 

See claim 4. Schelling teaches that the topic of interest (audio, video, text, etc.) is 
annotated using different styles (type indicators), (figs. 1, 4, 5). 
Claims 6 and 23: 

See claim 1 . Schelling teaches an indicator icon that indicates files containing sound and 
a text message describing the data file. Shelling illustrates text relating to (fig. 1) a sound 



I 
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recording of a person's (i.e. Grandma's) voice (col. 2, lines 62-67; col 3, lines 10-29). Lowitz 
teaches indication of audio information (col. 11, lines 62-67; col 12, lines 1-7). 
Claims 7 and 24: 

See claim 6. Schelling teaches that the system analyzes the objects (text, image, etc.), 
retrieves and displays the information (fig. 1; col. 2, lines 43-67; col. 3, lines 1-40; col. 4, lines 
57-67; col. 5, lines 1-14). 
Claims 10 and 27: 

See claim 7. Schelling teaches an index print having video frames extracted from video 
information and text information extracted from 
Claim 18: 

See claim 1. Schelling teaches a computer system having a processor, display, and 
memory (figs. 2 and 3; col. 3, lines 40-67; col. 4, lines 1-7). 
Claim 30: 

See claim 1. Schelling teaches an indicator icon that indicates files containing sound and 
a text message describing the data file. Shelling illustrates text relating to (fig. 1) a sound 
recording (audio information) of a person's (i.e. Grandma's) voice (col. 2, lines 62-67; col. 3, 
lines 10-29). 

Claim Rejections - 35 USC §103 
5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be 

patented and the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 2, 3, 19, and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Schelling/Choi/Lowitz and Nielsen et al (US 6,055,542). 

Claims 2 and 19: 

See claim 1 . Schelling does not teach that user input identifying a concept of interest is 
stored in a user profile. However, Nielsen discloses a system and method for allowing a user to 
create an interest profile for finding information of interest within a document. Nielsen explains 
that once the profile is created, it may be used on any document or web page to sort the 
information on the page according to the user's interests; once sorted, the information is 
displayed to the user (abstract; col. 1, lines 59-67; col. 2, lines 10-22). Therefore, it would have 
been obvious to one having ordinary skill in the art at the time the invention was made to modify 
Schelling' s system of retrieving and displaying information to include Nielsen's teaching of 
creating an interest profile because as Nielsen explains, it allows the user to view the points of 
interest without having to search the entire document. 
Claims 3 and 20: 

See claim 1 . Schelling teaches printing a matter descriptor such as a title identifying the 
subject matter of the data file (col. 5, lines 1-3). Schelling does not 

teach a relevance indicator for indicating a degree of relevance of the multimedia information. 
However, Nielsen teaches an interest profile that can include additional data such as relative 
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importance (weights); points of interest are sorted according to relevancy and are presented into 
the document as links (abstract; col. 1, lines 50-67; col 2,lines 1-9). Thus, it would have been 
obvious to one ordinarily skilled in the art at the time of invention to include Nielsen's teaching 
of indicating a degree of relevancy in Schelling's multimedia document because as Nielsen says, 
it allows the user to quickly spot the areas of interest within a document and move to a desired 
point of interest within the document or page (col. 4, lines 57-67; col. 5, lines 13-14). 

7. Claims 8, 9, 25, 26, and 31 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Schelling/Choi/Lomtz and Gibbon etal(US 6,098,082). 

Claims 8, 9, 25, and 26: 

See claim 1 . Schelling does not teach that the multimedia document includes printed 
closed-caption text information. However, Gibbon discloses a method for providing a 
compressed rendition of a video program in a format suitable for electronic searching and 
retrieval on the WWW. Gibbon teaches pictorial transcripts that are compact representations of 
video programs which are automatically generated by selecting representative frames or images 
from the video program and combining them with a second media component such as audio or 
text which is associated with each representative frame (abstract; col. 1, lines 55-67; col. 2, lines 
1-15; col. 3, lines 10-15). Gibbon teaches that a printed rendition of closed-captioned text may 
be provided. The printed rendition is a pictorial transcript in which each representative frame is 
printed with a caption containing the portion of the closed-caption text corresponding to the 
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scene from which the representative frame is taken (col. 3, lines 16-22). Thus, it would have 
been obvious to a person having ordinary skill in the art at the time of invention to modify 
Schilling to include Gibbon's teaching of printing closed-caption text because it provides a 
printable visual presentation of the sound associated with the image (frame) of interest; therefore, 
close captioning is not only visible on a TV receiver designed to display it but it is also visible 
when being printed on paper. 1 
Claim 31: 

See claim 8. Gibbon teaches that a printed rendition of closed-captioned text may be 
provided. The printed rendition is a pictorial transcript in which each representative frame is 
printed with a caption containing the portion of the closed-caption text corresponding to the 
scene from which the representative frame is taken (col. 3, lines 16-22). 

Conclusion 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to X L. Bautista whose telephone number is (571) 272-4132. The 
examiner can normally be reached on Monday-Thursday 8:00AM-6:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Weilun Lo can be reached on (571) 272-4847. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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9. Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




X L BaAxtistaV-^ 
Primary Examiner 
Art Unit 2179 
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July 7, 2005 



